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METHOD FOR SIMULTANEOUS BODY PART DISPLAY 

CROSS REFERENCE TO RELATED APPLICATIONS 

[0001] This application claims the benefit of a priority under 35 USC 1 19 to 
French Patent Application No. 00 16 957 filed December 22, 2000, the entire contents 
of which are incorporated by reference. 

FIELD OF THE INVENTION 

[0002] The invention relates to the display of digital images of body parts and 
more particularly to simultaneous display of the breasts of a patient obtained using 
mammography apparatus. 

BACKGROUND OF THE INVENTION 

[0003] Mammography apparatus is used for examining patients' breasts using 
X-rays. The image is acquired using an image receiver formed either by a 
photographic plate or by digital sensing means. Examinations practiced with such 
apparatus comprise taking cranio-caudal and lateral images Cranio-caudal views are 
obtained by irradiating the breast from above so as to obtain a view in the axis running 
from the patient's head to her feet. To obtain a side view, the breast is irradiated from 
the side so as to obtain a view in an axis passing through the patient's body. In both 
cases, or for views taken in other directions, it is useful for the practitioner when 
establishing the diagnosis to compare the images of both of the patient's breasts taken 
in the same direction. Practitioners often put the films of each breast on the 
negatoscope side by side in order to compare the images of the two breasts. 
Comparison notably allows structural disorganization within the breast to be located. 

[0004] A digital mammograph known as the Senographe 2000D is 
commercially sold by the assignee of the application. Image display protocols are 
provided for image display. In certain protocols, the images of both breasts in a given 
direction are displayed side by side. The mammograph also has a "fit to breast" 
function. This function allows, for a given image, an enlarged view of a region of 
interest of the breast to be displayed. This region of interest is constituted by the 
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smallest rectangle of the image that contains the complete breast. Margins are 
nevertheless provided around the breast. One problem encountered with this apparatus 
is that display does not necessarily allow ready comparisons of the images of both 
breasts. 

SUMMARY OF THE INVENTION 

[0005] There is consequently a need for a type of display providing simple 
and practical comparison of the two images obtained in examination apparatus. This 
problem is particularly prevalent for images of both breasts obtained in a 
mammograph. 

[0006] Briefly, in one embodiment of the present invention, the regions of 
interest of both images are employed for displaying images on which the regions of 
interest are aligned. Display of the two images now allows simple and direct 
comparison of both images. In the case of images of both breasts, the display allows 
immediate comparison of each breast. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0007] Figure 1 shows diagrammatically images of both breasts displayed in a 
known apparatus; 

[0008] Figure 2 shows diagrammatically images of both breasts displayed in 
one embodiment of the invention; 

[0009] Figure 3 shows diagrammatically other images displayed embodiment 
of the invention; 

[0010] Figure 4 shows diagrammatically images displayed according to 
another embodiment of the invention; 

[001 1] Figure 5 is a flow chart for an image display method; and 

[0012] Figure 6 shows a display apparatus. 
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DETAILED DESCRIPTION OF THE INVENTION 

[0013] The embodiments of the invention will be described with reference to 
its preferred application to the display of digital images of both breasts supplied by 
mammography apparatus. These images can be images acquired digitally for 
subsequent digital processing for their display. The images can also be obtained from 
analog radiography apparatus by exposure and development of films. The analog 
images can then be digitized and displayed as discussed below. 

[0014] Figure 1 shows diagrammatically images 2 and 4 of both breasts 
displayed in a known apparatus. The first image 2 shows the right-hand breast of a 
patient in a cranio-caudal view. The second image 4 shows the patient's left-hand 
breast in a cranio-caudal view. Both images are displayed side by side, the first image 
being to the left of the second image. The respective contours 6 and 8 of the breasts 
are shown diagrammatically on their images 2 and 4. On the images 2 and 4, regions 
of interest 10 and 12 defined around the breasts can also be seen. As explained above, 
each region of interest is formed by a rectangle surrounding the breast. This rectangle 
has, taking into account a margin surrounding the breast, a small as possible surface 
area. In of Figure 1, the views of the breasts on the first and second images are offset, 
The right-hand breast shown on the first image is higher up than the view of the left- 
hand breast on the second image. The "height" dimension with reference to Figure 1 
as well as to the remaining drawings refers to a dimension on the images taken along 
their adjacent edge. Because of the offset, it is difficult for the practitioner to compare 
the two images. The offset can simply originate from the manner in which each breast 
was positioned when the images were taken. The offset can also originate from a 
difference in size of the two breasts. 

[0015] Figure 2 shows diagrammatically images of the two breasts 
displayed according to one embodiment of the invention. As in Figure 1 , the images 
are two cranio-caudal images, and the same reference numerals have been used to 
identify the same elements. Figure 2 also shows the region of interest surrounding the 
breast on each image; it can be seen that the region of interest is at the same height on 
each image. In other words, the regions of interest of the left- and right-hand images 
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are aligned. Consequently, as Figure 2 shows, the two breasts are displayed at the 
same height on the two images. The right-hand breast shown on the first image 2 is at 
the same height as the view of the left-hand breast on the second image 4. Such a 
display facilitates the comparison of the images of both breasts. 

[0016] The display shown in FIG. 2 can be obtained as follows. Images of 
both breasts are acquired. jAs explained above, these may originate directly from 
apparatus fitted with a digital acquisition unit, or from a unit for digitizing analog 
images. A region of interest is defined on each image around the view of the patient's 
breast on the image. Definition of such a region of interest is described in the 
following European patent applications: EP- A- 1,047,0 18, EP-A-1, 035,507 or EP-A-0, 
912,963. Definition of such a region of interest is performed on the image using 
digital image processing techniques. Once the region of interest has been defined on 
each image, the images are displayed so that the regions of interest are aligned. As 
Figure 2 shows, for regions of interest of the same size in the vertical direction, 
alignment is performed by displaying the upper or lower edge of the regions of 
interest of the images at the same height. 

[0017] The digital images can be displayed on the same display apparatus 
such as a cathode ray tube, a liquid crystal display or any other type of display device. 
The images can also be displayed on two separate display devices side by side. 
Whatever type of device is used for display, the two digital images are displayed side 
by side' for comparison purposes wherein the regions of interest are aligned to 
simplify comparison. 

[0018] Figure 3 shows diagrammatically other images displayed according to 
an embodiment of the invention. The images on Figure 3 are side view images of a 
patient's breasts in a mammograph; the same reference numerals have been employed. 
Figure 3 shows that the breast shown in the first image 2, to the left of the figure, is 
larger than the view of the breast on the second image 4 of Figure 3. This size 
difference can simply originate from differing manipulations when taking the images. 
In the embodiment of Figure 3, the right-hand breast may have been positioned 
differently from the left-hand breast when the images were taken. Two arrows 12 and 



4 



14XZ00155 



14 can also be seen on the images 2 and 4 and these indicate the respective positions 
of the tip of the breast on each image. In the embodiment of Figure 3, the position of 
the tip of the breast is used for aligning the regions of interest on the two images. The 
position of the tip of the breast can be determined using known image analysis 
techniques. For the alignment, the height in the image of this tip of the breast can 
simply be used. More generally, where the regions of interest of both images are not 
of the same size in the vertical direction, one can, for alignment purposes, optimize 
one image-dependent criterion, as a function of the relative height of the images. Such 
a criterion can be the result of computing correlation over the whole of the region of 
interest, or over a part of this region of interest. One could also proceed to correlate 
images in the area adjoining the tip of the breast. One could also proceed to align the 
contour of a breast or part of the contour of one breast with respect to the other breast. 
Such computations allow determination of the relative height for which this criterion 
is at its maximum; this height now corresponds to alignment of the regions of interest 
on both images. The result, shown in Figure 3, is that the images are, like in Figure 2, 
displayed so as to facilitate their comparison. The technique described with reference 
to Figure 3 is particularly useful when the regions of interest determined for the two 
images are not of the same size. They are now aligned by calculating an optimization 
criterion which depends on the relative position of the images, after which this 
criterion is optimized. 

[0019] Figure 4 shows diagrammatically images displayed in another 
embodiment of the invention. In the embodiment of Figure 4, not only are the regions 
of interest aligned, but also the images are enlarged. The same magnification factor is 
applied to both the images to still ensure ready comparison of each image. The 
magnification factor is calculated as a function of the size of the region of interest on 
the first image and the size of the region of interest on the second image. The 
magnification factor, common to both images is selected so that the region of interest 
of each image is wholly contained within each displayed image. In other words, the 
enlargement enables the complete breast to be seen on the image displayed. In the 
example of Figure 3, the region of interest on the image of the right-hand breast 
occupies a greater area than the region of interest on the image of the left-hand breast. 
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The magnification factor to obtain the display shown in Figure 4 is calculated using 
the region of interest of the first image. In Figure 4, the region of interest of the first 
image consequently touches the upper and lower edges of the image. In contrast, for 
the same magnification factor, the region of interest of the second image is not 
touching the edges of the second image. Both images are nevertheless displayed with 
the same magnification factor which facilitates their comparison. This technique also 
applies to regions of interest of the same size. 

[0020] Figure 5 is a flow chart for carrying out the display method, for the two 
images. Steps 18-22 concern the first image and steps 19-23 the second image; and 
steps 24 to 32 constitute joint processing of both images. In a first step 18, the digital 
image of a breast is acquired. As explained above, acquisition can be of any type. 
Acquisition can comprise the application of numerous processing operations to the 
images, for example sensor correction processing operations, and operations for 
thickness compensation, automatic contrast setting or otherwise. At the next step 20, a 
region of interest is defined in the image. Next, at step 22, it is determined whether 
enlargement of the image is desirable. If so, processing passes to step 24 and, in the 
opposite case, to step 28. Steps 19, 21 and 23 correspond to steps 18, 20 and 22 for 
the second image. At step 23, like step 22, it is determined whether enlargement of 
the image is desirable and if so, control passes to step 24 or, in the opposite case, to 
step 28. Showing the flow chart with two separate branches is simply designed to 
demonstrate that image processing in these steps is independent. Processing can 
obviously be performed successively using identical apparatus as would be the case 
for images taken successively in the same apparatus. 

[0021] Response at steps 22 and 23 can be pre-programmed for a given 
display protocol or can result from user input. At step 24, a magnification factor is 
calculated. This factor is a minimum of the ratio between image size and region of 
interest size. The minimum on both images, in both directions is considered. The 
magnification factor obtained will ensure that the region of interest of each image is 
wholly contained within the enlarged image. At the next step 26, each image is 
enlarged by applying the calculated magnification factor. Control then passes to step 
28 where the two regions of interest are aligned. At step 13, both images are displayed 
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with their regions of interest aligned. The process then stops at step 32. It can 
obviously be repeated for other images. 

[0022] Figure 6 shows image display apparatus. The apparatus has a 

unit for acquiring digital images 34. This unit is for example an image sensor of 
digital apparatus, or a unit for digitizing analog images or, yet again, a unit for 
receiving storage media containing a digital image. The acquisition unit could also be 
simply comprised of storage media such as a hard disk. The acquisition unit supplies 
an image processing unit 36 with at least two digital images intended for simultaneous 
display. Image processing unit 36 processes the images and applies the processed 
images to a display device 38. As explained above, the display device can be of any 
type whatsoever. Image processing unit 36 can consist of a PC (personal computer), 
an image processing board including a microprocessor, or any other digital computing 
means known per se. This image processing unit comprises several separate 
processing blocks. These blocks are for example logic blocks in the processing unit. 
The first block 40 is responsible for defining a region of interest and defines, in each 
image received, a region of interest and outputs an image having a region of interest. 
The images supplied are fed to block 42 for enlargement. At block 32 a common 
magnification factor for the images is calculated; the images are then enlarged. The 
enlarged images are applied to block 44 handling alignment. As arrow 46 shows, the 
images with their region of interest can be applied directly to alignment block 44, 
bypassing the magnification block. In the alignment block, the regions of interest of 
the images are aligned. The output is aligned images which are applied to display 
device 38 for simultaneous display. This diagram does not show processing 
operations which may be applied to the images but which have no bearing on the 
method described. 

[0023] Obviously, the invention is not limited to the preferred embodiments 
discussed above; in particular, it can apply to images other than those of the two 
breasts in a mammograph; it can also apply to displaying of images of the same breast 
taken at different moments. 
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[0024] The alignment modes discussed with reference to Figure 2 for cranio- 
caudal images also apply to the side views in Figures 3 and 4 or, yet again, to any type 
of image. Inversely, the alignment modes of Figures 3 and 4 apply also to images 
consisting of cranio-caudal images. The region of interest in the example is 
rectangular but it could have any other shape. In the examples, the enlargement step is 
performed before the alignment step; but it can also occur after the alignment step. 
The examples refer to specific images; the method disclosed applies to other images 
as well. The method has also been described based on an example of two images but 
it can also apply to more than two images. Thus, two cranio-caudal images could be 
displayed simultaneously along with two side view images. In this case, the images 
could be enlarged using the same magnification factor for the four images. The 
images could also be aligned pairwise. The examples discuss images displayed side 
by side. The method can also be applied to images displayed one above the other 
with the dimension of interest no longer being the vertical dimensions, but rather 
horizontal dimensions. 

[0025] Various modifications in structure and/or steps and/or function may be 
made by one skilled in the art without departing from the scope and extent of the 
invention as recited in the claims. 
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